Skip to content

Add generic WebDAV metadata cache - #646

Closed
taglia wants to merge 1 commit into
internxt:mainfrom
taglia:pr/webdav-generic-cache
Closed

Add generic WebDAV metadata cache#646
taglia wants to merge 1 commit into
internxt:mainfrom
taglia:pr/webdav-generic-cache

Conversation

@taglia

@taglia taglia commented Jul 26, 2026

Copy link
Copy Markdown

Summary

This adds a short-lived generic metadata cache for WebDAV item lookups and folder listings.

The cache is used by WebDAV handlers for repeated PROPFIND, GET, HEAD, PUT, DELETE, MOVE, and MKCOL lookup paths, and is invalidated on write operations.

Why

WebDAV clients often issue repeated metadata requests for the same paths during backup and restore workflows. Without a short-lived cache, these requests repeatedly hit Drive metadata APIs even when the state has just been fetched or mutated by the same WebDAV process.

Behavior

  • Caches Drive file/folder metadata and folder contents briefly.
  • Invalidates affected paths and parent folder listings after mutations.
  • Keeps existing WebDAV semantics unchanged.
  • Does not add client-specific path matching or backup-system-specific behavior.

Tests

  • Updated WebDAV tests to clear cache state between tests.
  • Verified WebDAV handler coverage for lookup and mutation paths.
  • Verified targeted auth/WebDAV tests locally.

@taglia
taglia requested a review from larryrider as a code owner July 26, 2026 10:46
@larryrider

Copy link
Copy Markdown
Contributor

hey @taglia!
One concern about this pr: today we already have a lightweight cache via DriveItemRepository (SQLite), but it only caches the path -> uuid mapping. Every lookup still validates the item against the backend, so we never report a file or folder as existing unless the backend confirms it.

WebDavCacheService changes that behavior by serving cached metadata and directory listings for up to 2 minutes without any backend validation. Invalidation only happens for changes made through this same WebDAV instance, so changes from the web app, another device, or another CLI/WebDAV instance could leave us serving stale ("phantom") entries until the TTL expires. I think that's a regression compared to the current behavior.

My main concern is that as we need to keep validating cache hits against the backend to guarantee correctness, I'm not sure this cache provides enough benefit over what we already have today. I'd like to hear your thoughts before deciding whether to merge this PR 🤔

@taglia

taglia commented Jul 29, 2026

Copy link
Copy Markdown
Author

Hi @larryrider,

Thanks, that makes sense. I agree that serving cached metadata/listings without backend validation can introduce phantom entries when Drive is modified outside this WebDAV instance.

Given that correctness trade-off, I do not think this PR should be merged as-is. The two other PRs cover the safer generic improvements, and I’ll retest performance with only those changes before deciding whether a narrower cache proposal is still useful.

If we revisit this, I’d constrain it to an approach that preserves backend validation semantics rather than returning authoritative cached entries.

I will close this and test probably next week, I have a large backup in progress now and Hyper Backup can only run one at a time.

@taglia taglia closed this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants